Deterministic Annealing Variant of the EM Algorithm
نویسندگان
چکیده
We present a deterministic annealing variant of the EM algorithm for maximum likelihood parameter estimation problems. In our approach, the EM process is reformulated as the problem of minimizing the thermodynamic free energy by using the principle of maximum entropy and statistical mechanics analogy. Unlike simulated annealing approaches, this minimization is deterministically performed. Moreover, the derived algorithm, unlike the conventional EM algorithm, can obtain better estimates free of the initial parameter values.
منابع مشابه
An average case performance of the deterministic annealing EM algorithm
Average case performance of the deterministic annealing EM algorithm is evaluated for Gaussian mixture estimation problem under some additive noises. The data-averaged EM update equations with respect to hyperparameters are calculated analytically in the large data limit. We find that the EM algorithm strongly depends on the initial conditions. Moreover, by using our analysis, it becomes possib...
متن کاملAnnealing Techniques For Unsupervised Statistical Language Learning
Exploiting unannotated natural language data is hard largely because unsupervised parameter estimation is hard. We describe deterministic annealing (Rose et al., 1990) as an appealing alternative to the ExpectationMaximization algorithm (Dempster et al., 1977). Seeking to avoid search error, DA begins by globally maximizing an easy concave function and maintains a local maximum as it gradually ...
متن کاملDeterministic Quantum Annealing Expectation-Maximization Algorithm
Maximum likelihood estimation (MLE) is one of the most important methods in machine learning, and the expectation-maximization (EM) algorithm is often used to obtain maximum likelihood estimates. However, EM heavily depends on initial configurations and fails to find the global optimum. On the other hand, in the field of physics, quantum annealing (QA) was proposed as a novel optimization appro...
متن کاملDeterministic annealing EM algorithm in parameter estimation for acoustic model
This paper investigates the effectiveness of the DAEM (Deterministic Annealing EM) algorithm in acoustic modeling for speaker and speech recognition. Although the EM algorithm has been widely used to approximate the ML estimates, it has the problem of initialization dependence. To relax this problem, the DAEM algorithm has been proposed and confirmed the effectiveness in small tasks. In this pa...
متن کاملNBCT: A Toolkit for Naive Bayes Clustering
3 Clustering algorithms 5 3.1 ML/MAP based clustering . . . . . . . . . . . . . . 5 3.1.1 Parameter estimation based on ML . . . . . . 5 3.1.2 Parameter estimation based on MAP . . . . . 5 3.1.3 Membership distribution . . . . . . . . . . . 6 3.1.4 Dissimilarity . . . . . . . . . . . . . . . . . 7 3.1.5 Clustering . . . . . . . . . . . . . . . . . . . 7 3.1.6 Relevance analysis . . . . . . . . ...
متن کامل